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ABSTRACT 



This paper describes the results of research conducted on the perceptions of college and 
university presidents relative to the perceived impact of the institutional research office on 
institutional effectiveness at their respective institutions. This investigation focuses on the 
presidents’ perception of effectiveness of some of the basic activities central to institutional 
research, planning, and assessment. Among them, reporting of data to internal and external 
institutional constituents, institutional planning support, student need and opinion evaluation, 
academic program planning and assessment, enrollment projection and analysis, and student 
learning outcomes assessment. Institutional research units were also evaluated relative to the 
degree to which they are reactive vs. proactive, provided timely and accurate data and 
information, demonstrated understanding of data and information needs of campus decision 
makers, and the degree to which the president relied on the institutional research unit. 
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Overview 

A cursory examination of the institutional research literature reveals a wealth of material 
addressing the institutional research professional’s self-perceived impact on institutional 
effectiveness. Nedwek and Neal (1994), Harrington (1994), Meredith (1994), Matier (1994), 
and Clagget and Huntington (1993) have cited the place and importance of institutional research 
on institutional well-being and in the assessment of institutional effectiveness. Saupe (1989), 

- Taylor (1990), and Nichols and Wolfe (1990) have argued additionally that to this end, 
institutional research should play a prominent role in the assessment of student learning outcomes 
and educational effectiveness. However, little purposeful research focuses exclusively on the 
perceptions of the institution’s chief executive officer. Regardless of the location of the 
institutional research office in the organizational hierarchy, it is the college or university 
president that ultimately determines the role, function, and context of institutional research and 
planning activities in organizational management and strategic planning. 

The depth and breadth of activities undertaken by institutional research offices are likely 
to differ, as dictated by individual circumstances such institutional size, mission, location of the 
office in the organizational hierarchy and locus of control (Chen, 1992), (Harrington, Knight, 
and Christie, 1994), (Harrington and Chen, 1995). 

The functional activities of institutional research cover a vast array of topics which 
typically touch upon nearly every comer of the academy. Examples of such efforts include 
studies conducted on enrollment management, retention, student and alumni opinion, budget 
development and deployment, operational and strategic planning support, academic program 
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review, evaluation of instruction, and facility utilization. Institutional research provides data and 
information in support of reporting requirements for internal and external institutional 
constituents. Institutional research engages in environmental boundary scanning activities, which, 
serve as effective early warning systems to alert institutional decision makers of impending 
institutional vulnerabilities. The recent explosion of information technology has greatly 
enhanced the timeliness, utility, and usefulness of such activities. 

The degree to which institutional research has a direct impact on issues of institutional 
governance and planning is based largely on the location of the office in the organizational 
hierarchy (Volkwien, 1989), (Chen, 1992), (Harrington, Knight, and Christie, 1994), and 
(Harrington and Chen, 1995). The location of the office in the organizational structure affects 
its ability to function effectively, influence the nature of institutional research activities 
undertaken, and determine the importance and impact on the institution’s decision support 
system. Given this assertion to be accurate, the authors conducted a study of college and 
university presidents to determine the degree to which the perceptions of institutional research 
practitioners, and those of university presidents, converge on issues of the effectiveness of 
institutional research activities in strategic and operational planning, and assessment. The results 
of this study provide an overview of the perceived effectiveness of institutional research activities 
by institutional type and location of the institutional office in the organizational hierarchy. 
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Methodology 

In early January 1996, the authors mailed surveys to a random sample of four hundred 
eighty five (485) college and university presidents. The survey response rate was 53.9% (261 
respondents). The survey instrument elicited demographic data on institutional control (public 
vs. private), institutional type, Fall 1995 enrollment, FY96 institutional budget, the tenure in 
length of years of the principle institutional research officer, and the individual to whom the 
institutional research office reports. Data were collected relative to degree to which the 
institutional research office is perceived as reactive or proactive, the degree to which the IR 
office should be reactive or proactive, how frequently the president relies on the Institutional 
research office, presidential expectations of institutional research, and the perceived effectiveness 
of the institutional research activities of general institutional data reporting, institutional research 
studies, strategic planning, and student learning outcome assessment activities. 

Characteristics of the Respondents 

Geographic Location of Respondents 

Survey responses were submitted from 47 U.S. states, including Alaska and Hawaii. The 
largest number of responses came from presidents at colleges and universities in New York (28), 
followed by Texas (15), Pennsylvania and Florida (12 each), and California (11). It is asserted 
by the researchers that the number of responses from each state represent the opinions and 
perceptions of college and university presidents in general. 
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Type and Control 

Data were collected on institutional type (two-year, four-year, university) and control 
(public vs. private). The largest number of respondents (31.4%) are presidents of public 
universities. Presidents of four-year private institutions accounted for 24. 1 % of all respondents. 
The fewest responses came from presidents of private two-year institutions (2.0%). 

Size 

Instiutional size was determined by fall semester or quarter student enrollment. 

The mean Fall 1995 headcount enrollment for all respondent institutions was 9,613. 

Public universities had the largest mean enrollments at 18,621, while private two-years 
institutions had the smallest mean fall headcounts at 439. 

Operating Budget 

The mean fiscal year 1996 operating budget for all institutional respondents 
was $ 176.9 million. Public universities budgets average $475.7 million, followed by private 
universities at $ 1 16.2 million. Public two and four year institutions indicated budgets of $ 27.5 
and $ 28.9 million respectively. Private two-year institutions had mean annual operating budgets 
of $ 1.6 million. 
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Centralization and Locus of Control of Institutional Research 

In part, the authors desired to determine the degree to which the institutional research 
function at U.S. colleges and universities is centralized and the locus of control for the unit. 

Of the four hundred eighty-five institutions surveyed, 77% responded that they had a centralized 
institutional research office on campus. There is a modest correlation between institutional type 
and the centralization of institutional research; the larger the institution (four-year - > university) 
the greater the likelihood that the institutional research function is centralized. Of those 
acknowledging a centralized office, nearly half (45.9%), of the institutional research 
professionals report to a Vice President for Academic Affairs. Universities and four-year 
institutions typically have a larger proportion of their institutional research professionals 
reporting to a Vice President for Academic Affairs, whereas institutional research professionals 
at two-year institutions most frequently r°port directly to the institution’s president. 

Proactive vs. Reactive Institutional Research 

A central focus of the research project was to ascertain presidential perception of the 
degree to which the institutional research unit is aggressive in matters of internal and external 
environmental boundary scanning activities in support of college planning and decision making 
needs for data and information. Respondents were asked to identify whether they perceived their 
institutional research offices as proactive or reactive in responding to various institutional needs. 
Half of the individuals responding (50.3%) indicated that they perceived their institutional 
research offices as reactive. A greater percentage of presidents at public universities perceived 
their institutional research offices are reactive than did presidents of four-year and two-year 
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institutions. Presidents of public institutions perceived their institutional research offices as less 
proactive than those offices in private institutions. A greater percentage of institutional research 
practitioners reporting directly to the institution’s president were perceived as more proactive, 
than those that reported elsewhere in the organizational hierarchy. Institutional research 
professionals reporting to a Vice President for Academic Affairs were more likely to be 
perceived as reactive than their peers reporting to other campus leaders. When asked whether 
they preferred their institutional research functions to be proactive or reactive, 89.2% indicated 
a preference for proactive institutional research. 

Reliance on Institutional Research 

The authors postulate that the perceived effectiveness of the institutional research unit on 
a given college or university campus is largely dependent upon the degree to which the 
institution’s chief executive officer relies on the unit for data and information support. College 
and University presidents were asked to indicate the degree to which they relied on their 
institutional research units for support. Of those responding to the survey, more than half 
(53.4%) indicated that they relied "often" on their institutional research unit. Presidents of 
universities relied more heavily on their institutional research unit, than did their peers at four- 
year or two-year institutions. Fewer than seven percent (7%) of all respondents indicated that 
they "rarely" or "never" relied on their institutional research staff. Institutional research staff 
reporting directly to the institutions president were relied upon more heavily than their 
colleagues who reported elsewhere in the institution’s leadership structure. 
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Length of Service 

The authors theorize that the degree to which an institutional research unit is effective, 
is in part attributable to the institutional researcher’s familiarity with the institution and the 
environmental context in which it operates. In assessing the perceptions of presidents regarding 
institutional research, we also collected data on the number of years of institutional service held 
by institutional researchers. The mean years in service for all institutional research professionals 
was 6.07. Institutional researchers at public four-year institutions have the longest tenure at 
9.18 years,, followed by those at public universities at 6.93 years. The shortest tenure is found 
at private four-year institutions, where instiutional research professionals have been in their 
current positions for four years. Institutional research officers reporting to a Vice President for 
Academic Affairs had the longest tenure at 6.97 years. Institutional Research staff reporting to 
a Vice President for Planning had the shortest tenure at 4.36 years. 

Presidential Expectations of Institutional Research 

A section of t!.e survey instrument was devoted to collecting data related to presidential 
expectations of their institution’s institutional research function. Seven specific areas were 
addressed: the importance of assignments, accuracy of data and information, timeliness and 
relevance of data and information, analysis and interpretation of data, conclusions and 
recommendations accompanying institutional research reports, experience of institutional research 
staff, and the ability of institutional research staff to understand the information needs of campus 
decision makers. 
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Assignment of Tasks 

Two-thirds of the presidents responding to the survey indicated that the tasks they assign 
to the institutional research office at their institution are extremely important. Presidents at two- 
year and four-year institutions placed greater importance on tasks assigned to institutional 
research than did their peers at universities. The importance of tasks and institutional type were 
negatively correlated; the larger the institution, the less importance placed on tasks assigned to 
the institutional research office. Significant relationships were found between the importance 
placed upon tasks assigned by an institution’s president and their reliance on the instiutional 
research unit. The more frequently the president relies on the institutional research office, the 
greater importance placed upon the tasks assigned. [ANOVA df(main) = 3, df(within) = 194, 
(F = 7.8864, sig. = .0001] 

Accuracy and Integrity of Data 

Among college and university presidents, 96 % either agreed, or agreed strongly that their 
respective institutional research offices provided accurate data and information. Presidents of 
four-year institutions indicated greater satisfaction with the accuracy of institutional research data 
than their peers at either two-year colleges or universities. Only four percent of presidents were 
not satisfied with their institutional research office’s ability to provide accurate data. There is 
a modest correlation between president’s satisfaction with the accuracy of data and the degree 
to which the institutional research office provides proactive decision and planning support . The 
more proactive the institutional research office, the greater the president’s confidence in the 
reliability and validity of institutional data. Significant relationships were found between the 
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perception of accuracy of data by college presidents and their reliance on the instiutional 
research unit. The more frequently the president relies on the institutional research office, the 
greater confidence placed in the accuracy and integrity of data provided by the instiutional 
research office. [ANOVA F = 6.8319, df(main) = 3, df (within) = 194, sig. = .0002] 



Timely and Relevant Information Support 

Presidents were asked to rate their level of agreement with the statement “Proactively, the 
institutional research office offers timely, relevant information”. The majority of respondents 
(49.4%) indicated general agreement with the statement. Nearly one-third agreed strongly. One 
in five college or university presidents either disagreed, or disagreed strongly with the statement 
of timely and relevant information support. Presidents of universities were likely to be critical 
of the information support offered by their institutional research offices. Chief executive officers 
of 4 year colleges indicated greater levels of satisfaction with the timeliness and relevancy of 
information support provided by their office of institutional research than were their colleagues. 
There is a modest correlation between president’s satisfaction with the timeliness and relevancy 
of information support and degree to which the institutional research office provides proactive 
decision and planning support . The more proactive the institutional research office, the greater 
the president’s confidence in the timeliness and relevance of institutional data and information. 
A modest correlation exists also between satisfaction with the timeliness and relevance of 
information support and the level of reliance placed upon the institutional research office by the 
president. The greater the reliance placed upon the institutional resefch office by the 
institution’s president, the greater the level of his or her satisfaction with the timeliness and 
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relevance of the data and information. Significant relationships were found between the 
perception of providing proactive and relevant data and ^formation to the institution’s president 
and their reliance on the instiutional research unit. The more frequently the president relies on 
the institutional research office, the greater confidence placed on the timeliness and relevance of 
data provided by the instiutional research office. [ANOVA F = 17.5607, df(main) = 3, 
df (within) = 192, sig. = .0000] 



Analysis and Interpretation of Institutional Data and Information 

When asked if they expected analyses and interpretations to accompany institutional 
research reports, ninety percent of the respondents indicated that they did expect such. Ten 
percent of the respondents indicated that they did not expect analysis or interpretation of 
institutional research reports. Presidents of universities and two-year colleges had greater 
expectations for analysis and interpretation of reports, than did presidents of four-year institutions. 

Recommendations for Institutional Action 

Although presidents of colleges and universities have a high level of expectation for 
analyses and interpretation of institutional research reports and studies, nearly one-third do not 
expect or require recommendations to accompany institutional research reports. Twenty-nine 
percent of the respondents articulated an urgency for recommendations to accompany reports. 
Presidents of universities typically had a greater expectation for recommendations for institutional 
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action to accompany institutional research reports. Presidents of four-year institutions indicated 
the least need for such institutional research support. 

Understanding Information Needs of Campus Decision Makers 

Presidents were asked to assess the degree to which institutional research practitioners on 
their campuses understood the information needs of key campus decision makers. Presidents of 
two -and four-year institutions indicated a greater level of satisfaction with the level of 
understanding exhibited by their institutional research professionals, of the data and information 
needs of key campus decision makers. Presidents of universities responded less enthusiastically. 
Modest correlations were found between the degree to which the institutional research office was 
proactive, the degree to which the president of the institution relied on the office, and the level 
of understanding campus decision support needs. Presidents viewed those proactive offices on 
which they relied heavily, as being more attuned to the data and information support needs of 
campus decision makers. Significant relationships were found between the president’s perception 
of the degree to which their institutional research unit thoroughly understands the information 
needs of campus decision makers and the president’s reliance on the instiutional research unit. 
The more frequently the president relies on the institutional research office, the greater the 
confidence placed in the units understanding of information needs of decision makers. 
[ANOVA F = 9.4429, domain) = 3, df(within) = 192, sig. = .0000] 
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Assessment of Institutional Research Effectiveness 

The third section of the survey solicited presidential perception of the effectiveness of the 
institutional research unit relative to student learning outcomes assessment, reporting to external 
and internal constituents, strategic planning support, the ability to gauge student opinion, 
academic program assessment and planning, enrollment projection and analysis, evaluating 
instructional effectiveness, and conducting space and facility utilization reports and studies. 

Student Learning Outcomes Assessment 

One third of president’s surveyed perceived their institutional research offices as being 
very effective in assessing student learning outcomes. Presidents of four-year and two-year 
institutions were predisposed to perceive their institutional research staff as more effective at 
learning outcomes assessment than presidents of universities. In regards to locus of control, 
institutional research staff reporting directly to the institutions president were perceived as more 
effective than their peers who reported elsewhere. Significant relationships were found between 
the president’s perception of effectiveness in assessing student learning outcomes and the 
president’s reliance on the instiutional research unit. The more frequently the president relies on 
the institutional research office, the greater the perception of effectiveness. [ANOVA F = 
5.6552, 

df(main) = 3, df(within) = 182, sig. = .0010] 
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Data Reporting to Institutional Constituents 

College and university presidents perceive their institutional research staffs as very 
effective in reporting accurate and timely data to internal and external institutional constituents. 
A greater proportion of presidents at four-year institutions perceived their institutional research 
staff as more effective in reporting data and information to constituents than their peers at two- 
year institutions or universities. There is a modest correlation between the degree to which the 
institution’s president relies on the institutional research office and their effectiveness in reporting 
data and information to institutional constituencies. Institutional research staff who reported to 
either the institution’s president or vice president for academic affairs were perceived as more 
effective than their peers that reported to other campus administrators. Institutional research 
officers reporting to a vice president for academic affairs were perceived as more effective in 
reporting data and information to external constituents, whereas those reporting to the institution’s 
chief executive officer were perceived as more effective reporting data and information to internal 
constituents. Significant relationships were found between the president’s perception of 
effectiveness in reporting accurate, timely information to institutional constituents and the 
president’s reliance on the instiutional research unit. The more frequently the president relies on 
the institutional research office, the greater the perception of effectiveness. [F = 5.6552, 
df(main) = 3, df(within) = 191, sig.=.0010] 
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Institutional Planning Support 

The majority of presidents responding to the survey perceive their institutional research 
offices as somewhat effective in providing support for institutional planning activities. Nearly 
one-in-four presidents perceive their institutional research offices as ineffective. Presidents of 
universities perceive their institutional research offices as less effective in providing institutional 
planning support than do presidents of two-year and four- year institutions. President’s at four- 
year institutions perceive their institutional research offices as more effective than their peers. 
There is a correlation between the perceived effectiveness of institutional planning support and 
the degree to which the institution’s president relies on the instiutional research staff, as well as 
the degree to which the institutional research office is proactive. The more proactive and relied 
upon, the greater the perceived effectiveness in planning support. Institutional research staff 
reporting to a vice president for planning were perceived by college and university presidents as 
more effective than their peers who reported elsewhere in the organizational hierarchy. 
Significant relationships were found between the president’s perception of effectiveness in 
strategic planning support, the president’s reliance on the instiutional research unit, and 
institutional type. The more frequently the president relies on the institutional research office, 
the greater the perception of effectiveness in planning support. [ANOVA F = 5.9660, 
df(main) = 3, df( within) = 185, sig. = .0031]. Presidents of two-year institutions placed greater 
confidence in their instiutional research office’s effectiveness in strategic planning support than 
did presidents at four-year colleges and universities. [ANOVA F = 12.9831, df(main) = 2, 
df(within) = 186, sig. = .0013]. 
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Student Need and Opinion Evaluation 

Presidents of two-year institutions perceive their institutional research professionals as 
more effective in monitoring and evaluating student need and opinion than do presidents of four- 
year institutions and universities. Nearly one-third of university presidents perceive their 
institutional research staffs as ineffective. Modest positive correlations exist between the degree 
to which the institutional research office is proactive and effective in student learning outcomes, 
and president’s perceived effectiveness in monitoring and evaluating student need and opinion. 
Institutional research staff reporting to an institution’s president were perceived as more effective 
in monitoring and evaluating student need than those reporting to either vice presidents of 
academic affairs, planning, student affairs, or business affairs. Significant relationships were 
found between the president’s perception of effectiveness in gauging student opinion, the 
president’s reliance on the instiutional research unit, and institutional type. The more frequently 
the president relies on the institutional research office, the greater the perception of effectiveness 
in monitoring student opinion and need. [ANOVA F = 3.4145, df(main) = 5, df(within) = 176, 
sig. = .0187]. Presidents of two-year institutions placed greater confidence in their instiutional 
research office’s effectiveness in strategic planning support than did presidents at four-year 
colleges and universities. [ANOVA F = 4.6017, df(main) = 2, df(within) = 179, sig. = .0113]. 

Academic Program Assessment and Planning 

Seventy-five percent of college and university presidents perceive their institutional 
research staffs as effective in conducting academic program assessment and planning activities 
on campus. Presidents of two-year institutions perceive their institutional research staff as more 
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effective than do either presidents of four-year colleges or universities. There are correlations 
among the degree to which the institutional research office is proactive, relied upon, effective in 
reporting data and information to internal and external constituents and effective providing 
institutional planning support, and their perceived effectiveness in assisting in academic program 
assessment and planning. President’s whose institutional research staff reported to either the vice 
president for academic affairs or directly to the president, were perceived as more effective than 
their colleagues reporting elsewhere. Significant relationships were found between the president’s 
perception of effectiveness in academic program review and planning, the president’s reliance on 
the instiutional research unit, and institutional type. The more frequently the president relies on 
the institutional research office, the greater the perception of effectiveness in 
academic program review and academic planning. [ANOVA F = 2.8246, df(main) = 3, 
df(within) = 177, sig. = .0176]. Presidents of four-year institutions placed greater confidence in 
their instiutional research office’s effectiveness in strategic planning support than did presidents 
at four-year colleges and universities. [ANOVA F = 7.3007, df(main) = 3, df(within) = 182, 
sig. = .0001]. 

Enrollment Projection and Analysis 

Institutional research activities of enrollment projection and analysis were perceived as 
effective by half of the presidents surveyed. Twenty percent indicated the their staffs were 
ineffective in such endeavors. Presidents of four-year colleges perceived their institutional 
research staffs as more effective at enrollment projection and analysis than presidents at either 
two-year institutions or universities. There is a correlation between the degree to which the 
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institutional president relies on the instiutional research office, the degree to which the 
institutional research office is effective in institutional planning activities, the degree to which the 
institutional research office is effective in providing data and information support to internal 
institutional constituents, and president’s perceived effectiveness of institutional research in 
providing enrollment projections and accompanying analyses. Institutional research staff 
reporting directly to a vice president for planning were perceived as most effective in support 
enrollment projections and analyses. Significant relationships were found between the president’s 
perception of effectiveness in developing useful enrollment projections and the president’s 
reliance on the instiutional research unit. The more frequently the president relies on the 
institutional research office, the greater the perception of effectiveness and utility. 

[ANOVA F = 7.8872, df(main) = 3, df(within) = 181, sig. = .0001] 

Assessment and Evaluation of Instruction 

Fewer than ten percent of the college and university presidents surveyed perceived their 
institutional research offices as very effective in the assessment and evaluation of instruction. 
Forty-percent of the respondents indicated that their institutional research offices were somewhat 
effective. Presidents of two-year institutions perceived their institutional research office as more 
effective than did presidents of four-year colleges and universities. Despite convention, 
institutional research staff reporting to a vice president for academic affairs were perceived as less 
effective at evaluating instruction than those reporting directly to the institution’s president. 
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TABLE 1 
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Presidential 


Item Mean 


Agreement 


Expectation 


Rating 1 


Index 2 


Importance of Tasks Assigned 


3.586 


98.0 


Accuracy of Data 


3.460 


95.5 


Timely, Relevant Data/Information 


3.071 


80.1 


IR Has Great Deal of Experience 


2.984 


76.7 


1R Understand Information Needs 


2.898 


71.9 



1 Item Mean Rating - mean rating obtained from use of a 4 point likert-type scale to 

ascertain expectation measure (4 = strongly agree, 1 = strongly disagree). 

2 Agreement Index - measure indicates the percentage of all respondents who indicated "Agree" 

or "Strongly Agree" with importance of institutional research activity. 
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Institutional Research Effectiveness Measures 
TABLE 2 



Institutional Research 
Activity 


Item Mean 
Rating 1 


Effectiveness 

Index 2 


Outcomes Assessment 


3.043 


79.6 


Internal Reporting 


3.508 


91.1 


External Reporting 


3.505 


93.3 


Strategic Planning Support 


3.016 


77.2 


Gauging Student Opinion and Need 


2.940 


73.6 


Academic Program Assessment 


2.961 


74.6 


Enrollment Analysis/Projections 


3.281 


85.4 


Evaluation of Instruction 


2.416 


52.2 


Facility Utilization Reporting 


2.661 


62.1 



1 Item Mean Rating - mean rating obtained from use of a four point likert-type scale to 

ascertain effectiveness measure (4 = effective 1 = ineffective). 

2 Effectiveness Index - measure indicates the percentage of all respondents who indicated that 

institutional research activity item as "Effective" or "Somewhat Effective" 
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Barriers Hindering the Effectiveness of the Institutional Research Function 

The final section of the survey solicited presidential perception on possible barriers to the 
effectiveness of the institutional research function on their campuses. The two greatest perceived 
impediments to the effectiveness of institutional research professionals were increased government 
reporting requirements (IPEDS, Student-Right-to-Know,) and lack of institutional financial 
support for the institutional research unit. Also mentioned frequently were lack of initiation of 
institutional research professionals, and lack of campus-wide cooperation with the institutional 
research office. 

Summary 

The role and function of institutional research are important components to institutional 
well-being and are critical in monitoring and evaluating institutional effectiveness. Regardless 
of the nature of institutional control or mission, presidents rely heavily on institutional research 
to provide data, information, and planning and assessment support. Effective, formal institutional 
research activities are conducted in virtually every corner of the academe, from student learning 
outcomes assessment to institutional planning support, and from academic program assessment 
and evaluation of instruction to student affairs research. Although burdened to provide 
continuous support to an ever increasing array of internal and external institutional constituencies, 
institutional research offices continue to be severely understaffed and under-supported budgetarily. 

Even though college and university presidents realize the conditions under which their 
institutional research pro'cssionals work, institutional research offices are perceived as less 
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proactive than presidents would like, although little is done to improve institutional researcher’s 
lot. 



College and university presidents perceive a number of barriers to the effectiveness of 
institutional research, including increased federal and state government reporting requirements and 
lack of internal financial support for the institutional research unit. 
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